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ABSTRACT 

Although there are many reasons to evaluate training 
and development (T & D) programs, it is often time-consuming and 
expensive to do so. However, evaluations can be conducted by an 
auditing approach. The auditing approach to evaluating T & D offers 
three advantages. Auditing is quicker and less expensive than 
traditional methods, and it uses a format that is easily understood 
by managers. Auditing can be used to pinpoint areas where further 
examination is warranted. Repeated auditing also serves as an 
incentive for producing high quality T & D efforts. One form of T & D 
auditing is a third-party evaluation based on critical questions and 
samples of information that fall into five categories: compliance, 
process, operations/financial, trainees, and business results. The 
audit process requires six steps: (1) establish objectives for each 
category of the audit; (2) describe what the auditor expects to find; 
(3) determine data collection procedures; (4) collect data samples; 
(5) report what was found in relation to expectations; and (6) write 
and report conclusions^ Auditing* s greatest value to the organization 
is in providing an unbiased third-party view of how, when, and where 
T & D contributes to the organization. (A siumnary of an audit of a 
supervisory training program in a manufacturing firm is provided as 
an example. ) (KC) 
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There are many reasons to engage iu evaluating training and 
development (T & D). One reason would be to keep a complete, accurate, 
and detailed accounting of the satisfaction, learning, and performance 
resulting from every T & D program (Swanson & Sleezer, 1987). This is 
typically the view of the evaluation expert who sees evaluation as central to 
the T & D process. This view is usually expensive in terms of time and 
money. 

One explanation for the lack of formal evaluation in T & D departments 
(Parker, 1985) may be the high price that professionals must pay to live up 
to an evaluation ideal that compels T & D managers to evaluate all cases on 
all dimensions. This ideal, detailed, and comprehensive view of evaluation 
has its place, but it is not the only reasonable perspective on T & D 
evaluation. Practices in the accounting profession provide an excellent 



opportunity to examine an alternative. Accountants are known to be 
meticulous when it comes to recording every minute expenditure of an 
organization. Yet, one of the most common roles accountants engage in is 
auditing, a method of sampling financial records. Auditing is a process that 
produces an official accounting and verification, most often conducted by 
third party evaluators. Furthermore, auditing typically relies on samples of 
information that are critical to the organization and its decision makers. 
An auditing of T & D departments and/ur programs provides an 

'f' 

evaluation without incurring the high expenses of traditional evaluation. It 
also provides an unbiased third-party view which helps managers make 
reasonable decisions regarding how, who. when, where, and how much 
T & D contributes to the organization. The T & D audit presented here is a 
third-party evaluation that is based on critical questions and samples of 
information that falls into five categories. 

• COMPLIANCE: Where does T & D satisfy the knowledge and skills 
required within the organization? 

• PROCESS: Is T & D effective in applying commonly accepted, 
professional practices to accomplish its work? 

• OPERATIONS/FINANCIAL. Is T & D efficient in its operation, as well 
as its use of human, capital, and financial resources? 



3 



• TRAINEES: Does T & D achieve its objectives in terms of trainee 
satisfaction, iearnitig, and performance 

• BUSINESS RESULTS: Does T & D produce economic and social gains 
for the orgaoization, and if so. what are the cost-benefits on the T&D 
investment? 

AUDIT CATEGORIES 

The following sections provide explanations and examples for each of 
the five T 6e D audit categories and the following diagram shows the 
relationships between audit categories (Figure 1), Also, the diagram 
graphically portrays the general format of the audit. Typically, the audit is 
conducted in an organization by taking samples in each categcuy. 
Cofflpliance Audit 

The purpose of the Compliance Audit is to determine what operating 
requirements, policies, rules, regulations, or industry norms apply to 
employee knowledge and skills. Policies and regulations can be examined at 
industry, company, and department levels. Sampling is conducted to 
dctermme if employees possess required knowledge and skills, especially 
where primary corporate objectives and regulations apply. 
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Training and Deueiopment Audit 
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Why? Because every company has skill needs based on its business 
plans, the status of its work force, and the requirements it must meet from 
external agencies such as the government When skills are lackmg. the 
company must either acquire them by hiring new people or build them by 
traming existing employees. And that is where T&D fits into the picture. 
How well can or does T&D satisfy these needs? 

The Compliance Audit examines needs by referencing the 
organization's plans and policies rather than its existing T&D programs. The 
Compliance Audit looks for critical areas where employee development 
should be instituted or enhanced. The emphasis is on locating areas of risk 
or liability m the organization, where employee knowledge and skills are an 
absolute must 
£xaixu2l£ 

• An organization has policies about how candidates are screened and 
selected for employment. Certain criteria must be met such as 
asking legally defensible interviewing questions and meeting 
affirmative action goals. The audit reviews personnel hiring 
records and employment litigation to determine if managers 
have the skills to follow selection and hiring policies. 
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• The local fire marshall requires an organization to inform 
employees about proper evacuation procedures. The audit chooses a 
fair sample of emplo- 3s to determine if they know how to leave the 
building in a safe and orderly manner. 

• Government regulations require an organization to train employees 
in the proper handling of hazardous materials. The audit scans 
safety training records to determine if employees have received 
both initial and updated training on the hazardous materials they 
use on a regular basis. 

• A T&D department requires supervisors to submit a critique of 
employee performance after subordinates attend a training program. 
The audit comp ires returned critiques against the number of 
attendees. 

Process Audit 

The purpose of the Process Audit is to evaluate the framework used to 
implement programs and services. Factors such as project management 
steps, quality control methods, and reporting procedures are reviewed. 

Why? Effective T&D usually follows some routine procedures to 
produce programs and provide services. Process steps and controls are 
used to specify who is to do what and when. Examining these processes 

Er|c 8 



highlights critical activities which affect the probability of producing 
successful training efforts. 

The audit looks for procedures which assure that the correct tasks are 
performed in the appropriate order. Existing procedures are compared to 
commonly accepted professional practice. The audit format most often used 
for evaluating T&D processes is listed below (Swanson. 1987). 

• Analyze 

• Design 

• Develop 
e Deliver 

• Evaluate 
£xmcl££ 

• A T&D department often receives requests from line managers to 
develop specific skill building programs. The audit examines the 
nature of requests and how the department responds. The presence 
of discrete analysis praaices is checked to determine whether 
training and non-training factors were identified. The use of general 
and problem-specific analysis procedures is examined. 

• An organization produces customized media programs for 
organization-specific topics. The audit looks at design practices and 
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documentation to assess if learning variables and instructional 
sequencing were analyzed and the media selected. 
• In an organization, many subject matter eiperts deliver certain 
highly technical programs. The audit reviews how instructional 
materials are assembled and presented. Factors such as how 
trainers prepare materials for experts and how experts present 
complex subjects is examined. 
Operations/Financiafl Audit 

The purpose of the Operations Audit is to determine how efficiently 
T&D activities are planned and executed. Efficiency is calculated by 
comparing work output with available and allocated resources. Resource 
categories include staff, facilities, materials, and equipment Emphasis is 
placed on how T&D is implemented, not on programs. 

The operations audit compares results produced with the level of 
resources applied. A systems approach is used to explore how management 
accomplishes its objectives across several dimensions such as supervision, 
work planning and controls, reward systems, continuity of operations, and 
response times. 

A personnel department engages in numerous T&D projects iind 
ERIC 10 
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administers a broad range of programs. The audit checks for planning atd 
control systems that are used to guide and coordinate multiple activities. 
Monitoring and follow-up procedures are also reviewed. 

The purpose of the Financial Audit is to determine how allocated 
resources (funds, staff time, facilities, materials, and equipmeAt) are 
consumed by T&D. The Financial Audit examines financial planning, 
reporting, and control procedures. In addition, the audit reviews how 
spending decisions are made. It also examines purchasing patterns by line 
item and by program. 

Why? Attention that is focused on the financial activities of business 
offers many useful indicators of performan^. Cost control is a major issue 
for most organizations, especially the costs in overhead. The general value 
o( activities, such as T&D, can be measured in meaningful terms. 
Eimsls. 

Each year employees submit budget requests for projects to be 
developed and implemented in the following year. Dollar amounts on each 
budget are identified and justified. The audit looks ai planned versus actual 
T&D spending. It reviews direct and indirect expenditures for T&D projects 
to estimate true costs. The audit also reviews the timing of expenditures, 

especially toward the end of each fiscal year, 

o 11 
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Trainees Audit 

The purpose of the Trainees Audit is to assess how weJJ T&D programs 
and services achieve objectives. Here, the audit compares program nnd 
service outputs with their stated objectives. Only existing T&D activities are 
reviewed. Auditing in this area is not intended to be neither a need.i 
assessment nor a means to determine if an activity is warranted at all. 
Rather, this audit checks the effects a program or service has on its clients. 

The Trainees Audit is accomplished by sampling data that measures 
satisfaction, learning, and performance. Five groups can offer input — 
trainees, their peers, their supervisors, their subordinates, and their 
customers. Evaluation information comes from data-gathering methods 
such as skill demonstrations, questionnaires, group and individual 
interviews, observations, and records analysis, 
Eiamoles 

• Each T&D session concludes with participants completing a program 
evaluation form. Leaders summarize the data and periodically 
rep(X't results. The audit samples evaluation sheets to search Soc 
response patterns and then interviews previous trainees to confirm 
findings. 

• In an organizations career counseling services are offered. 
ERIC 1 p 
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Employees can obtain advice about developmental programs and 
how to make career decisions. The audit samples employees to 
determine the long-term effects of such services. 
• Many skills learned by tramees are applied to interactions with 
customers. The audit reviews skill objectives that apply to customer 
relations and samples employees who have received training. Then, 
customers are sampled to determine if the skills are applied. 
Business iiesuits Audit 

The purpose of the Business Results Audit is to determine the return 
on investment for programs and services. Break -even and cost-benefit 
analyses are selectively applied to gage the value of outputs. Estimates ?re 
made to judge the effects of increasing, maintaining, or decreasing the 
amounts of resources to be applied to sampled programs and services. 

Why? T&D programs and services are provided to develop employees 
to meet business needs. The company wants to develop skills in individuals 
and groups. Generally, company managers expect these activities to 
f ^duce useful results. They assume that investing in T&D produces 
specific economic and/or social gains. 

The Business Results Audit uses ezistmg organization standards to 
establish reference points for comparisons on "returns." Also, the Business 

o 
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Results Audit uses findings from other T&D audit categories to establish 
reference points for "investments," From these data, the relative value of 
programs and services can be identified. Emphasis is placed on the 
relationship between critical business needs and employee development 
issues. Finally, aaion plans for the future may be identified. 
Eiamples 

• A company experiences traditionally high turnover in clerical jobs. 
The cost of hiring and training a single clerk is estimated to be 
$1,800. An orientation program was developed to encourage new 
clerical staff to value the benefits of long-term employment with the 
company. The audit finds that departments that use the orientation 
program experience, on average, 6X lower turnover. A cost-benefit 
analysis reveals a break-even point (cost recovery) for program 
development after five months of orientations. 

• Stress management training is conducted for all customer service 
representatives to help them cope with irate customers. Stress is the 
most often cited complaint for these employees. The audit samples 
former attendees and discovers that stress levels remain unchanged. 
Further, evidence reveals that representatives do not have ready 
access to customer information. A recommendation is made to offer 

o 
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the program on a per request basis only. Also management j 
encouraged to review the procedures by which representatives 
access customer information. 

AUDIT PROCESS 
The audit process is quite simple and straightforward. The process 
requires that the evaluator work through six steps. 

• Establish objectives for each category of audit 

• Describe what he or she expects to find 

• Determine data collection procedures 

• Collect data samples 

• Report what was found in relation to expectations 

• Write and report conclusions 

A recent audit of a Supervisory Training (ST) program in a 
manufacturing firm resulted in a fifteen-page report The executive 
summary of that repoii't follows: 
Supervisory Training Audit: Eiecutive Summary 

A comprehensive audit/evaluation of the Supervisory Trammg (ST) 
was conducted to determine whether or not: 1) the ST is meeting the 
knowledge and skill requirements of Apex's first-line supervisory staff; 
2) the ST is effective (doing the right thmgs) in terms of accepted training 
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and development practices; 3) the ST is efficient (doing things right) in 
terms of program administration; and 4) the ST trainees are satisfisd with 
the program, how much trainees are learning, and the degree to which 
trainees are using their new skills and knowledge on the job. The audit also 
attempted to determine business results of the ST in terms of return on 
investment and (probable, possible) social gains. 
Supervisory Training Findings and Conclusions 

The audit revealed that the ST is meetmg the skill and knowledge 
requirements (as defined in the ST needs assessment) of first-line 
supervisors and that ST course recos\mendations from the needs 
asssessment have been or are being implemented. [However, when the ST 
course outline was compared with a sampling of employee relations policies 
there were some discrepencies.) 

An exammation of the ST documentation revealed that the ST program 
has and is using commonly accepted training and development practices, 
i.e., a needs assessment was conducted, ST courses were designed directly 
from the analysis work and ST courses were systematicaJly developed, 
implemented and evaluated. 

The ST is operated very efficiently. Employee Relations and Training 
and Development work very well together. Critical decisions are made by 

ERIC 16 
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Employee Relations and Training and Development managers and 
professionals, while administrative and clerical tasks are handled by 
capable clerical personnel. 

Overall, ST trainees are very satisfied with the ST program. However, a 
number of supervisors have expressed an interest in more technical types 
of courses. 

Program instructors and participants estimated that approximately 
two-thirds of ST course objectives/concepts are learned by participants. 
However, instructors were slightly more optimistic than supervisors. 

Shop supermtendents and line workers reported that supervisors do 
use the skills and knowledge gained from ST on the job. A variety of critical 
incidents were cited to substantiate these reports. 

Supervisors were not able to report any lime savings resulting from 
their participation in ST. However, they were able to report a number of 
positive critical work incidentf^ indicating changes directly related to their 
participation in the ST. These critical incidents have had considerable 
financial impacts on the Corporation. Given these financial impacts 
associated with the ST. it is estimated that the break -even point was 
reached at the end of one year. 
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Supervisory Training Recommendations 

• Continue to offer the ST courses presently listed in the ST course 
outline. Consider offering course updates for those ST courses that 
deal with volatile subject matter such as equal employment 
opportunity, product liability, and labra* law/employee relations. 
Examine personnel and other administrative policies that are not 
being addressed through the ST and implement a course of action to 
ensure proper coverage of appropriate organizational policies in the 
ST. 

f Future ST course specifications should stress evaluation more 
stringently. All ST courses should have some form of evaluation, 
including trainee satisfaction, learning, and performance. 

• As ST has reached a maturity stage, some thought should be given to 
starting ST II. To accomplish this, the training and development 
process should be recycled starting with the analysis phase. i.e., 
needs assessment, subject matter analysis, interviews, etc. 

• Given the per person costs for ST courses, it is not worth spending a 
lot of time looking for alternative vendors, unless prices rise 
dramatically. In fact, it will be difficult to find alternative vendors 
for courses such as Resources Management. 

ERIC 18 
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Supervisors see themselves as relativeiy experienced and very aware 
of the employee relations portions of their jobs. They feel that they need 
fewer employee relations types of courses and more technical courses. If ST 
II is offered, consider including more technical courses. 

Given the financial implications of participating in ST, participation 
should be required. Arrangements should be made for all supervisors who 
have not attended ST to make-up the course as soon as possible. 

SUMMARY 

The auditing approach to evaluating T&D offers these advantages. 
Auditing is quicker and less expensive than traditional methods, and uses a 
format that is easily understood by managers. The limitations of such 
audits include inaccuracies due to poor sampling techniques or the 
possibilities of "adversarial" audits being conducted for political reasons. 
However, auditing can be used to pinpoint areas where further examination 

I 

is warranted. Repeated auditing also serves as an incentive for producing 
quality T&D efforts. Auditing's greatest value to the organization is in 
providing an unbiased third-party view of how, when, and where T&D 
contributes to the organization. 
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